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EXECUTIVE  SUMMARY 


Carnegie  Mellon  University  (CMU)  is  conducting  a  research  program  for  the  Federal  Aviation 
Administration's  (FAA)  National  Aging  Aircraft  Research  Program  to  develop  robotic  tools  to 
assist  aircraft  inspectors  by  automating  the  collection,  archiving,  and  post-processing  of 
inspection  data.  The  results  of  this  program  will  establish  the  technical  feasibility  of  using 
robotic  nondestructive  inspection  (NDI)  systems  in  major  aircraft  maintenance  facilities.  This 
program  has  been  funded  through  the  William  J.  Hughes  Technical  Center.  USAir  is 
supporting  the  project  by  providing  technical  guidance  from  experienced  aircraft  inspectors  and 
access  to  aircraft  in  its  Pittsburgh,  PA,  maintenance  facilities. 

During  a  preliminary  phase  of  work  under  this  program,  CMU  studied  the  task  of  aircraft 
inspection,  compiled  the  functional  requirements  for  an  automated  system  to  inspect  skin 
fastener  rows,  and  developed  a  conceptual  design  of  an  inspection  robot.  The  purpose  of  this 
robotic  system  was  to  automatically  deploy  conventional  sensors  used  by  aircraft  inspectors. 
The  system  was  designed  to  be  sufficiently  flexible  to  allow  for  the  incorporation  of  new  sensor 
technologies,  including  those  being  developed  by  other  organizations  participating  in  the  FAA's 
National  Aging  Aircraft  Research  Program. 

A  prototype  of  the  robotic  inspection  system,  (the  Automated  Nondestructive  inspector  or 
ANDI),  has  been  developed.  The  first  phase  of  system  development  resulted  in  a  laboratory 
system  that  demonstrated  the  ability  to  adhere  to  the  surface  of  an  aircraft  panel  and  deploy  a 
standard  eddy-current  sensor.  Results  of  the  first  phase  are  documented  in  report 
DOT/FAA/CT-94/23,  June  1994. 

This  report  describes  the  most  recent  developments  in  the  program,  covering  the  period  from 
January  1 993  through  June  1 995.  The  major  accomplishments  of  this  phase  of  the  project  are: 

•  The  mechanics  of  the  inspection  robot  have  been  enhanced  for  mobility  around  the 
circumference  of  the  fuselage. 

•  Video  cameras  have  been  added  to  the  robot  to  enable  remote,  and  eventually 
automated,  navigation. 

•  An  onboard  computer  has  been  added  to  the  robot  for  sequencing  low-level  tasks, 
such  as  actuating  cylinders. 

•  Software  to  control  the  eddy-current  system  remotely  has  been  integrated  with  the 
system. 

•  Communication  between  the  ground  and  onboard  computers  has  been  installed. 

•  The  capability  to  overlay  a  wireframe  model  of  the  robot  over  selected  aircraft 
models  was  added  to  the  operator  interface  to  provide  graphical  navigational 
information  to  the  inspector. 

•  Automatic  alignment  algorithms  have  been  developed  and  tested  under  laboratory 
conditions.  These  algorithms  have  not  yet  been  integrated  with  the  system. 

•  The  robot  was  deployed  and  demonstrated  during  the  1994  Air  Transport 
Association  (ATA)  NDT  Forum  hosted  by  the  FAA’s  Aging  Aircraft  NDI  Validation 
Center  operated  by  Sandia  National  Laboratories  in  Albuquerque,  New  Mexico. 

Several  operational  limitations  of  the  robotic  inspection  system  were  exposed  during  the 
laboratory  testing  and  field  demonstration.  Specific  problems  include  this  prototype  robot’s 
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speed,  absence  of  a  high-level  of  operator  control,  and  mechanical  reliability.  As  a  result  of 
these  experiments,  insight  has  been  gained  into  means  for  improving  speed,  ease  of  operation, 
and  mechanical  performance.  Future  development  efforts  will  be  directed  towards  obtaining 
more  autonomous  robot  operation  by  providing  higher  level  operator  controls. 
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1.  INTRODUCTION 


Carnegie  Mellon  University  (CMU)  is  conducting  a  research  program  for  the  Federal  Aviation 
Administration's  (FAA)  National  Aging  Aircraft  Research  Program  to  develop  robotic  tools  to 
assist  aircraft  inspectors  by  autorhating  the  collection,  archiving,  and  post-processing  of 
inspection  data.  The  results  of  this  project  will  establish  the  technical  feasibility  of  using  robotic 
systems  in  aircraft  maintenance  facilities.  Operational  and  economic  issues  associated  with 
robotic  inspection  of  aircraft  are  beyond  the  scope  of  this  project. 

1.1  Background 

1.1.1  Envisioned  Inspection  System 

In  the  preliminary  phase  of  this  project,  a  conceptual  design  of  a  robotic  aircraft  inspection 
system  was  developed.  The  task  of  aircraft  skin  inspection  was  selected  as  the  first  application 
to  be  automated  because  the  airline  industry  indicated  a  preference  for  automation  of  this 
function.  The  task  is  to  look  for  small  cracks  around  fasteners  due  to  stress  fatigue  as  well  as 
for  thinning  of  the  skin  due  to  corrosion.  Searching  for  such  flaws  is  a  highly  repetitive  task  that 
may  lead  to  a  reduced  probability  of  detection  of  defects  due  to  inspector  fatigue  or  boredom. 

The  emphasis  of  this  effort  was  to  design  a  system  that  would  assist,  rather  than  replace, 
aircraft  inspectors.  The  inspection  tasks  would  be  divided  between  the  human  inspector  and 
the  robot.  The  automated  system  would  deploy  the  sensors  in  a  consistent  manner  and 
process  sensor  signals  for  any  abnormal  indications  while  the  inspectors  would  monitor  the 
system  and  would  be  required  to  make  final  judgments  on  unusual  sensor  readings. 

The  design  goals  for  the  system  were: 

•  Modular  -  Both  the  hardware  and  software  for  the  system  would  be  modular, 
allowing  for  relatively  easy  changes  to  the  system. 

•  Extensible  -  The  system  would  be  designed  to  allow  other  modules  to  be  added  to 
the  basic  architecture  as  needed,  thus  extending  the  system’s  capabilities. 

•  Easy  to  use  -  To  be  of  benefit  to  the  user  community,  the  system  would  have  to  be 
easy  for  users  to  operate. 

•  Computer  platform  independence  -  Multiple  organizations  will  have  a  variety  of 
computing  hardware  requirements.  Thus,  the  system  wouid  have  to  be  functional 
across  a  variety  of  computer  platforms. 

•  LAN  aware  -  Because  airlines  have  distributed  inspection  centers,  information  must 
be  shared  within  a  specific  inspection  facility  as  weli  as  across  inspection  faciiities. 
Tying  the  systems  to  a  local  area  network  (LAN)  would  provide  this  capability. 

Work  on  the  preliminary  phase  of  this  project  to  develop  a  conceptual  design  of  an  automated 
aircraft  inspection  system  was  initiated  in  May  1991 .  The  initial  work  comprised  reviewing 
airiine  inspection  procedures  and  state-of-the-art  robotics  technology  and  creating  a  conceptual 
design  of  a  system  cailed  the  Automated  Nondestructive  Inspector,  ANDI.  This  was  completed 
in  December,  1991.  The  basic  design  of  the  robot  consisted  of  the  following  elements  [1]: 

•  Mechanical  System  -  The  mechanical  device  was  envisioned  as  a  cruciform  robot 
that  would  adhere  to  the  aircraft  fuselage  using  suction  cups. 
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•  Control  System  -  Based  on  the  map  of  a  specific  aircraft  contained  in  the  database 
and  the  inspection  to  be  performed,  the  control  software  would  generate  a  path  for 
the  robot  to  follow.  The  control  software  would  also  control  the  movements  of  the 
electro-mechanical  hardware. 

•  Data  Management  System  -  Information  on  the  types  and  locations  of  abnormal 
sensor  signals  would  be  stored  in  a  database.  The  database  would  contain  a  map 
of  the  surface  of  each  type  of  aircraft.  Each  specific  aircraft  would  have  an 
inspection  record  that  would  describe  and  map  the  locations  of  repairs  and  possible 
flaws.  The  system  would  gather  inspection  data  and  store  it  locally.  This  data  would 
be  shared  with  the  entire  maintenance  facility  as  well  as  with  all  of  the  carrier’s 
maintenance  facilities  nationwide. 

•  Sensors  -  The  robot  would  deploy  eddy-current  sensors  on  the  skin  to  test  for 
surface  and  subsurface  cracks  and/or  corrosion  which  causes  thinning  of  material. 
Small  onboard  cameras  focusing  on  the  fuselage  surface  would  provide  the 
inspector  with  images  of  the  skin  and  would  be  used  for  robot  navigation. 

•  Human-Machine  Interface  -  This  would  comprise  the  following  elements:  video 
monitors,  teleoperation  controls,  and  a  workstation  with  graphics  capabilities. 

Figure  1  depicts  the  system  architecture  which  was  envisioned  in  the  preliminary  phase  of  work. 

The  envisioned  system  architecture  was  a  complex  one  which  divided  the  tasks  among  several 
ground-based  computers  and  an  onboard  computer.  The  ground-based  processors  would 
control  the  major  system  tasks,  such  as  database  acquisition,  robot  control,  and  sensor  signal 
processing.  The  onboard  processor  would  control  the  onboard  hardware  and  the  sequencing  of 
tasks.  Some  of  the  highlights  of  the  envisioned  system  architecture  were; 

•  interprocess  communications  between  multiple  ground  based  computers  through 
local  area  network  (LAN) 

•  additional,  special  purpose  operator  interfaces 

•  prototype  inspection,  aircraft,  and  maintenance  databases 

•  video  for  close-up  rivet  inspection  and  wide  angle  robot/aircraft  monitoring 

•  lights  and  illumination  control  for  onboard  video 

•  additional  robot  sensors,  i.e.,  end  of  motor  travel  (EOT)  and  collision  detection 
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1.1.2  Early  Prototype  Inspection  System 

The  conceptual  design  which  was  specified  in  the  preliminary  phase  of  work  described  a  very 
robust  inspection  system.  The  strategy  employed  by  CMU  was  to  approach  the  system 
development  in  phases;  the  product  of  each  phase  would  be  a  system  with  a  useful  subset  of 
the  complete  system's  capabilities.  In  addition,  each  system  would  be  upwardly  compatible  with 
subsequent  systems. 

Work  building  a  prototype  of  the  cruciform  robot  design  and  developing  software  to  support  its 
motion  began  May  1992.  This  was  completed  in  January  1993.  The  accomplishments  in  this 
phase  were  [2]: 

•  Mechanical  System  -  The  first  prototype  of  the  robotic  system  was  developed  and 
tested.  The  prototype  weighed  approximately  30  pounds  (14  kg),  and  was  able  to 
adhere  to  a  surface  and  perform  a  scan. 
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•  Control  System  -  The  development  of  the  control  software  was  initiated  in  this 
development  phase.  The  software  controlled  the  scanning  and  walking  motions  of 
the  mechanical  device. 

•  Data  Management  System  -  The  inspection  data  acquisition  and  arialysis  functions 
used  by  the  system  during  this  phase  were  those  available  in  a  commercial  eddy- 
current  system. 

•  Sensors  -  The  mechanical  device  deployed  a  commonly  used  sliding  eddy-current 
sensor  over  a  row  of  fasteners.  In  addition,  experimentation  with  video  cameras 
began,  and  a  video  specification  document  was  written. 

•  Human-Machine  Interface  -  The  system  functions  were  controlled  by  a  personal 
computer  (PC)  workstation.  A  simple  menu-driven  interface  was  provided  for  the 
operator.  The  output  from  the  commercial  eddy-current  system  was  shown  on  a 
second  monitor. 

As  shown  in  Figure  2,  one  ground-based  PC  controlled  the  scanning  and  walking  motions  of  the 
robot  while  a  second  ground-based  PC  controlled  the  data  acquisition  functions  for  the  eddy- 
current  data.  There  was  no  communication  between  these  PCs;  data  acquisition  was  initiated 
manually  by  the  operator. 
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FIGURE  2.  FIRST  PHASE  SYSTEM  ARCHITECTURE 


The  robot  control  and  operator  interface  software  was  monolithic;  one  software  module  was 
written  for  both  the  interface  and  control  tasks.  The  eddy-current  data  acquisition  task  was 
controlled  by  commercial,  stand-alone  eddy-current  system  software. 

The  mechanical  device  was  tested  at  orientations  near  horizontal  and  at  modest  slopes. 
Enhancements  to  allow  the  mechanism  to  traverse  the  entire  fuselage  were  deferred  until  the 
next  phase. 


1.2  Overview  of  Current  inspection  System 

Since  January  1993,  the  robot  has  become  more  capable  in  all  aspects  of  operation.  The 
mechanics  have  been  upgraded  to  allow  the  robot  to  adhere  and  walk  in  all  orientations  on  a 
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fuselage  surface.  The  January  1 993  version  of  the  robot  was  limited  to  walking  at  orientations 
near  horizontal  or  at  modest  slopes.  Enhancements  to  allow  the  mechanics  to  traverse  the 
entire  circumference  of  the  central  fuselage  region  have  been  implemented. 

The  electronic  and  computer  system  architecture  has  been  upgraded  to  more  closely  follow  the 
originally  envisioned  architecture  and  is  shown  in  Figure  3.  A  multiple  computer  architecture 
has  been  employed  with  communications  between  the  individual  computers  using  both  point-to- 
point  and  LAN  connections.  A  computer  has  been  installed  onboard  the  robot  to  control  the 
low-level  sequencing  of  mechanical  operations  and  to  monitor  the  status  of  the  robot  in  real 
time. 


Ground  i  Onboard 
Based  Robot 


Previously,  the  system’s  software  was  monolithic  rather  than  modular.  In  the  present  version  of 
the  robot,  the  software  has  been  designed  to  be  modular,  incorporating  many  cooperating 
processes.  Multiple  coordinated  robot  control  and  operator  interface  processes  now  exist. 
Intuitive,  graphical,  computer  based  controls  for  robot  operation  and  sensor  interpretation  were 
added  to  simplify  the  use  of  the  system. 

The  commercial  eddy-current  software  was  replaced  with  custom  software  which  has  been 
integrated  with  the  other  components  of  the  system.  Acquisition  of  eddy-current  data  can  now 
be  synchronized  with  the  robot’s  movements  by  the  robot’s  control  process.  Inspection  data 
can  be  displayed  to  the  operator  or  archived  for  later  analysis. 
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Video  cameras  were  added  to  the  robot  to  enable  the  operator  to  navigate  the  robot  over  the 
surface  of  the  fuselage  and  align  the  robot  with  the  rivets  to  be  inspected.  Automatic  algorithms 
to  perform  the  alignment  task  were  developed  and  tested  under  laboratory  conditions;  however, 
they  have  not  yet  been  integrated  with  the  system.  The  cameras  also  provide  close-up  images 
of  the  rivets  being  inspected  and  visual  feedback  of  the  robot’s  state  to  the  operator. 

The  capability  to  overlay  a  3-dimensional,  wireframe  model  of  the  robot  over  selected  aircraft 
models  was  added  to  the  operator  interface  to  provide  navigational  information. 

Once  these  improvements  had  been  made  to  the  robotic  inspection  system,  it  was  felt  that 
additional  testing  of  the  system  outside  of  the  laboratory  would  be  beneficial.  The  robot  was 
deployed  and  demonstrated  during  the  1994  Air  Transport  Association  (ATA)  NDT  Forum 
hosted  by  the  FAA’s  Aging  Aircraft  NDI  Validation  Center  operated  by  Sandia  National 
Laboratories  in  Albuquerque,  New  Mexico. 

The  remainder  of  this  report  explains  the  current  inspection  system  in  greater  detail. 
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2.  MECHANICAL  SUBSYSTEM 


Since  January  1993,  the  capabilities  of  the  robot  have  been  extended.  As  the  system 
was  tested,  any  limitations  uncovered  were  noted  to  potentially  be  addressed  in  later 
phases  of  the  project. 


The  cruciform  robot  was  designed  to  adhere  to  surfaces  in  all  orientations  using  suction 
cups,  to  walk  across  a  curved  surface,  and  to  deploy  sensors  on  a  surface.  The  robot 
design  is  shown  in  Figure  4;  pneumatic  and  electrical  connections  are  not  shown  in  this 
figure. 


FIGURE  4.  DBA  WING  OF  THE  CRUCIFORM  ROBOT 

The  basic  mechanical  design  of  the  robot  is  identical  to  that  of  the  January  1993  robot 
which  is  outlined  in  [2].  The  robot  comprises  a  spine  assembly  and  two  bridges.  The 
spine  assembly  is  the  main  member  upon  which  the  bridges  move.  The  sensor  bridge 
possesses  dual  functionality.  It  is  used  to  deploy  the  eddy-current  sensor  during 
scanning  and  is  also  used  for  support  during  walking.  The  stabilizer  bridge  is  used  only 
for  support  during  walking,  although  later  it  might  have  an  inspection  role.  The  robot 
was  made  modular  in  this  phase,  meaning  that  each  of  the  three  subassemblies 
contains  all  of  the  components  (e.g.,  valves,  manifolds,  etc.)  necessary  for  its 
operations.  A  photograph  of  the  current  electro-mechanical  assembly  is  shown  in 
Figure  5. 
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FIGURES.  CURRENT  ROBOT 


2.1  Modular  Design 

Because  the  robot  is  a  research  prototype,  it  was  decided  to  make  the  hardware 
modular  to  enable  easy  modifications  for  experimentation.  The  January  1 993  version  of 
the  robot  was  not  moduiar,  and  it  was  difficult  to  make  changes  or  additions  to  the 
mechanical  system  without  a  major  overhaul  of  the  entire  robot.  In  a  modular  robot, 
each  of  the  bridges  and  the  spine  wouid  be  independent  mechanicaiiy,  pneumatically, 
and  electrically.  Thus,  one  could  remove  the  sensor  bridge  from  the  robot,  modify  it, 
test  it,  and  after  it  was  working  properly,  reattach  it  to  the  robot.  In  fact,  later  in  this 
section,  the  addition  of  a  set  of  rails  to  the  spine  assembly  will  be  described.  The 
modular  design  of  the  robot  enabled  this  change  to  be  made  with  a  minimum  of 
disruption  to  the  rest  of  the  robot  and  with  a  minimum  of  downtime. 

2.2  Leg  Design 

It  was  desired  that  the  robot  operate  in  a  variety  of  orientations,  simuiating  various 
positions  on  an  aircraft  fuselage.  For  the  robot  to  be  able  to  withstand  the  forces 
present  in  all  orientations,  its  legs  were  stiffened.  Two  enhancements  over  the  previous 
leg  designs  were  implemented:  the  suction  cup  fittings  were  redesigned  to  reduce  the 
tipping  moment  and  to  reduce  the  effect  of  the  maximum  shear  force  and  a  linear 
bearing  assembly  was  added  to  withstand  the  maximum  shear  force.  Figure  6  shows 
the  present  leg  assembly. 
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FIGURES.  LEG  ASSEMBLY 


The  suction  cup  from  the  previous  phase  was  a  commercially  available,  off-the-shelf 
item.  It  consisted  of  a  3-inch-diameter  siiicone  suction  cup,  an  aluminum  suction  cup 
fitting,  and  a  baii  and  socket  joint.  There  were  two  probiems  with  this  suction  cup:  the 
length  of  the  tipping  moment  arm  from  the  surface  to  the  ball  and  socket  joint  created  a 
iarge  tipping  moment,  and  according  to  the  manufacturer,  the  ball  and  socket  joint  was 
not  designed  to  withstand  a  shear  stress.  These  probiems  had  to  be  addressed  to 
prevent  a  failure  during  the  tests.  The  solution  chosen  was  to  design  and  fabricate  a 
new  suction  cup  fitting.  The  new  fittings  contained  an  integral  ball  and  socket  joint  and 
were  machined  from  P.E.T.,  a  iight-weight,  easily  machinable  plastic  material.  The  joint 
was  designed  to  withstand  the  expected  forces,  and  by  iowering  It  into  the  suction  cup 
fitting,  the  length  of  the  tipping  moment  arm  was  reduced,  thereby  reducing  the  tipping 
moment. 

The  leg  assembly  from  the  previous  phase  was  composed  of  the  suction  cup,  suction 
cup  fitting,  the  ball  and  socket  joint  discussed  in  the  previous  paragraph,  and  an  air 
cylinder.  Besides  the  problems  with  the  suction  cups,  there  was  also  a  problem  with 
attaching  the  air  cylinder  directly  to  the  suction  cup  assembly.  Exposing  an  air  cylinder 
to  shear  loads  over  a  prolonged  period  of  time  can  break  the  seal  around  the  rod, 
causing  air  leaks.  To  correct  this  problem,  a  linear  pillow  block  bearing  assembly  was 
added  to  the  leg  configuration.  The  shaft  of  the  bearing  assembly  was  attached  to  the 
ball  of  the  custom  ball  and  socket  joint  previously  described.  The  other  end  of  the 
bearing  shaft  was  connected  to  the  cylinder  rod  via  a  linear  coupler.  Thus,  in  this 
design,  ail  of  the  forces  act  upon  the  bearing  assembly,  and  the  cylinder  is  used  only  for 
motion.  The  air  cylinder  is  not  exposed  to  shear  loads,  thus,  protecting  the  seal  around 
the  rod. 

The  legs  on  the  spine  assembly  are  fixed  in  place.  A  standard  ^  -20  stainless  steel  stud 
is  attached  to  the  ball  of  the  custom  ball  and  socket  joint.  The  studs  are  mounted 


through  holes  at  the  ends  of  the  head  and  tail  beams  and  they  are  held  in  place  using 
locking  nuts. 

2.3  Rails 

When  the  robot  was  tested  on  surfaces  approaching  vertical,  the  linear  motors  that  drive 
the  bridges  along  the  spine  stalled.  When  the  system  was  rebuilt  to  make  the  robot 
modular,  weight  was  added  to  the  bridges.  The  center  of  mass  of  each  bridge  was 
raised  causing  the  torque  about  the  linear  motors  to  be  too  great.  The  torque  about  the 
motors  caused  the  air  gap  between  the  motor  and  spine  to  become  uneven,  thus 
stalling  the  motor.  To  correct  this  problem,  a  pair  of  rails  was  added  to  the  robot. 

The  rails  are  anchored  in  the  head  and  tail  beams  of  the  robot  and  are  parallel  to  the 
spine.  Each  bridge  was  equipped  with  two  linear  pillow  block  bearing  assemblies  which 
ride  along  the  rails.  Thus,  the  forces  caused  by  the  bridges’  weight  act  only  upon  the 
rails  and  their  associated  linear  bearings.  This  relieves  the  torque  about  the  linear 
motors  and  the  air  gap  remains  uniform,  thereby  solving  the  problem  of  linear  motor 
stalling. 

2.4  Onboard  Computer 

As  described  in  Section  3.1 ,  an  onboard  computer  was  added  to  the  robot.  The  printed 
circuit  boards  for  the  onboard  computer  are  housed  in  a  card  cage  on  the  head  end  of 
the  robot  as  shown  in  Figure  7.  A  commercial  card  cage  was  modified  and  mounted  on 
the  robot. 


FIGURE  7.  ONBOARD  PROCESSOR 


2.5  Tether 

A  tether  was  designed  to  keep  the  robot  from  falling  to  the  ground  if  the  robot's  suction 
is  lost.  The  basic  principle  behind  the  management  of  the  tether  is  to  pay  out  the  tether 
as  necessary  while  the  robot  moves  along  the  aircraft  panel;  if  the  robot  loses  adhesion 
and  starts  to  fall,  the  tether  must  be  locked  in  position  to  keep  the  robot  from  falling.  A 
standard  safety  device  known  as  a  retractor  was  used  as  the  tether.  The  retractor 
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works  much  like  an  automobile's  seat  belt.  With  low  acceleration,  the  cable  is  payed  out 
as  necessary;  however,  with  a  sudden  acceleration  the  mechanism  locks,  preventing 
any  more  cable  from  being  payed  out.  Custom  bracketing  was  designed  and  fabricated 
to  attach  the  tether  to  the  head  and  tail  beams  of  the  robot.  The  tether  is  secured  to  a 
trolley  that  runs  along  a  rail  suspended  above  the  area  where  the  robot  is  deployed. 

2.6  Marking  System 

The  first  concepts  for  a  marking  system  were  tested  during  this  phase  of  work.  The 
marking  system  was  designed  to  physically  mark  the  surface  where  abnormal  sensor 
signals  are  encountered,  allowing  the  inspector  to  locate  and  further  inspect  the  marked 
areas  at  a  later  time.  The  marking  system  contains  a  spring-return  cylinder  and  either  a 
china  marker  or  a  self  inking  stamp.  A  three-way  valve  controls  the  operation  of  this 
feature,  and  the  air  pressure  and  flow  to  the  cylinder  are  regulated  to  ensure  an 
appropriate  supply  of  air. 

2.7  Feedback  Switches 

To  give  the  robot  a  sense  of  self  awareness,  switches  were  added  to  provide  feedback 
when  actions  were  completed.  Thus,  when  an  action  such  as  the  extension  of  a  leg 
occurs,  an  end-of-travel  switch  provides  a  signal  to  the  control  software  to  indicate  that 
the  action  has  been  completed.  Once  the  software  has  confirmation  that  an  action  has 
been  completed,  the  next  action  in  the  sequence  can  be  executed.  If  no  signal  is  sent  to 
the  control  software,  the  next  action  is  not  executed.  This  is  in  contrast  to  the  previous 
phase  where  timing  loops  were  used  to  allow  enough  time  for  the  robot  to  complete  an 
action,  but  no  feedback  was  provided.  Even  if  an  action  was  not  completed,  the  next 
action  in  the  sequence  was  executed,  and  the  operator  was  required  to  notice  the 
problem  and  stop  the  robot.  With  the  switches,  the  robot's  gait  has  become  much 
smoother. 

Vacuum  detection  switches  were  also  added  to  the  robot  during  this  phase.  They  are 
used  to  detect  if  the  suction  cups  are  holding  to  a  surface.  The  vacuum  switches  have 
an  adjustable  threshold;  if,  for  some  reason,  there  were  a  vacuum  leak  around  a 
suction  cup  and  if  the  threshold  were  not  reached,  the  suction  cups  already  adhering  to 
the  surface  would  not  be  released.  These  switches  are  used  to  minimize  the  possibility 
that  the  robot  will  lose  adhesion  to  the  surface.  The  status  of  all  switches  is  shown 
symbolically  on  the  debug  interface  screen  described  in  Section  3.4.3.1. 

2.8  Air  Consumption 

The  air  consumption  of  the  January  1993  prototype  robot  varied  depending  on  several 
factors,  for  example,  the  number  of  suction  cups  activated  at  a  given  time.  The 
maximum  rate  of  air  consumption  for  the  robot  was  7,2  standard  cubic  feet  per  minute 
(SCFM)  of  air.  USAir  inspectors  had  indicated  that  the  robot  should  consume  no  more 
air  than  a  standard  pneumatic  tool,  about  5-10  SCFM  of  air.  The  January  1993  robot 
fell  within  this  range;  however,  many  standard  air  compressors  that  are  used  to  run 
pneumatic  tools  do  not  produce  7.2  SCFM  of  air.  The  Carnegie  Mellon  team  decided  to 
reduce  the  robot’s  air  consumption  to  a  maximum  of  6  SCFM  to  run  the  robot  from  most 
standard  air  compressors  and  to  minimize  pressure  loss  in  the  air  line  between  the  robot 
and  the  ground.  The  laboratory  compressor  produced  6.2  SCFM  at  100  psi. 
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The  air  consuming  parts  of  the  sensor  deployment  unit  and  the  pivot  lock  air  bearings 
were  eliminated.  The  pivot  lock  air  bearings  were  replaced  with  a  dry  bearing.  The 
airpot  of  the  sensor  deployment  unit,  which  consumed  0.6  SCFM  of  air,  was  replaced  by 
an  air  cylinder,  which  did  not  require  a  constant  draw  of  air.  On  the  spine  assembly,  to 
reduce  the  amount  of  torsion  about  the  front  suction  cup  and  its  ball  and  socket  joint,  a 
head  beam,  similar  to  the  tail  beam,  and  a  second  suction  cup  were  added.  Thus,  the 
spine  assembly  increased  from  three  to  four  suction  cups.  Accordingly,  another  ejector 
was  added  to  provide  suction  for  the  extra  suction  cup. 

The  maximum  rate  of  air  consumption  of  the  present  prototype  robot  is  5.6  SCFM  of  air. 
During  normal  operation,  only  four  suction  cups  are  affixed  to  the  surface.  The  normal 
requirement  of  air  is  3.2  SCFM  (four  suction  cups  and  the  linear  motor  air  bearings). 
When  the  robot  is  walking,  during  the  transition  from  the  spine  legs  being  affixed  to  the 
surface  to  the  bridge  legs  being  affixed  (or  vice  versa),  all  eight  cups  will  be  affixed  to 
the  surface  for  a  brief  moment.  The  moment  when  all  eight  cups  are  affixed  to  the 
surface  plus  the  constant  draw  of  the  linear  motors  gives  the  maximum  air  consumption 
of  5.6  SCFM. 
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3.  ELECTRONIC  AND  SOFTWARE  SYSTEMS 


3.1  Onboard  Robot  Electronic  Systems 

A  block  diagram  of  the  electronic  systems  carried  on  board  the  robot  is  shown  in  Figure 
8.  A  major  improvement  to  the  current  robot  is  the  addition  of  a  dedicated  computer 
carried  on  the  robot.  This  processor  manages  all  of  the  low  level  robot  control, 
feedback,  and  sequencing  in  real  time.  A  commercially  available  single  board  industrial 
computer  is  currently  used  for  the  onboard  computer.  This  processor  card  provides 
integrated  input/output  capabilities  over  a  -40°  to  85°C  operating  range.  Mechanical 
limit  switches  and  vacuum  switches  provide  TTL  level  signals  to  the  CPU  card  for 
monitoring  the  robot’s  status.  A  reed  relay  expansion  card  provides  the  12  VDC  outputs 
used  to  drive  the  solenoid  valves  for  operation  of  the  pneumatic  systems.  Commands 
are  received,  and  status  information  is  sent  via  an  RS-232  serial  communication  link 
with  the  ground  based  operator  workstation.  The  eddy-current  and  video  signals,  as 
well  as  the  motor  drive  signals  are  managed  directly  from  the  ground.  A  DC/DC  power 
converter  is  used  to  generate  the  5  VDC  power  required  for  the  computer  cards  from  the 
robot’s  normal  12  VDC  supply. 
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FIGURE  8,  ONBOARD  COMPUTER  AND  ROBOT  ELECTRONICS 
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The  onboard  computer  executes  a  multitasking  program  enabling  complex  sequences  of 
actions  to  be  simply  controlled  from  the  ground.  A  sample  task  called  “SmplExtendV”, 
short  for  “sample:  extend  robot  legs  with  vacuum”,  would  be  used  during  a  walking 
sequence  when  performing  the  transition  from  having  the  spine  attached  to  the  plane’s 
surface  to  having  the  bridge  legs  attached  and  extended.  The  steps  of  this  task  break 
down  to  the  operations: 

1 .  set  the  leg  cylinder  pressure  to  low 

2.  extend  the  bridge  legs 

3.  turn  on  the  bridge  vacuum  pumps 

4.  wait  until  all  four  bridge  suction  cups  have  achieved  vacuum 

5.  turn  off  the  spine  vacuum  pumps 

6.  wait  until  all  four  spine  suction  cups  have  lost  vacuum 

7.  complete  extension  by  setting  leg  cylinder  pressure  to  high 
To  perform  this  task  the  computer  would  execute  the  following  instructions: 

BB  NewTask  SmplExtendV 

WritePort  SensHPBypass  0  Endinstruct 
WritePort  StabHPBypass  0  Endinstruct 
WritePort  SensExtend  1  Endinstruct 
WritePort  StabExtend  1  Endlnstiruct 
WritePort  SensVacuum  1  Endinstruct 
WritePort  StabVacuum  1  Endinstruct 

SendMsg  SmplExtendV:  Waiting  for  bridge  vacuum  make 
Endinstruct 

ReadPortUntil  SensVacDetR  0  Endinstruct 
ReadPortUntil  SensVacDetL  0  Endinstruct 
ReadPortUntil  StabVacDetR  0  Endinstruct 
ReadPortUntil  StabVacDetL  0  Endinstruct 
WritePort  SpineVacuum  0  Endinstruct 

SendMsg  SmplExtendV:  Waiting  for  spine  vacuvim  break 
Endinstruct 

ReadPortUntil  SpineVacDetR  1  Endinstruct 
ReadPortUntil  SpineVacDetL  1  Endinstruct 
ReadPortUntil  SpineVacDetFR  1  Endinstruct 
ReadPortUntil  SpineVacDetFL  1  Endinstruct 
WritePort  SensHPBypass  1  Endinstruct 
WritePort  StabHPBypass  1  Endinstruct 
SendMsg  SirplExtendV:  Done  Endinstruct 
TaskDone  Endinstruct 
EE 

Additional  task  capabilities  include  timed  waits,  other  conditional  and  looping  constructs, 
and  the  ability  to  stop  or  start  other  tasks. 

3.2  Ground-Based  Electronic  Systems 

The  ground-based  electronic  equipment  consists  of  three  distinct  components:  the 
operator  workstation  computer,  the  video  processing  computer,  and  the  satellite 
equipment  enclosure. 
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3.2.1  Operator  Workstation  Computer 

The  operator  workstation  computer  is  the  main  control  point  for  the  robotic  system.  It 
provides  the  display  and  interface  resources  required  by  the  operator  to  control  and 
monitor  the  robot.  These  interfaces  are  described  in  further  detail  in  Section  3.4.3.  All 
high-level  robot  control  and  coordination  are  performed  by  this  machine. 
Communication  channels  are  provided  to  the  motor  controllers,  the  onboard  computer, 
and  the  video  processing  computers.  The  eddy-current  instrument  is  contained  in  and 
controlled  by  this  machine.  A  photograph  of  the  operator  workstation  is  presented  in 
Figure  9. 


FIGURE  9.  OPERATOR  WORKSTATION  &  VIDEOMONITOR/RECORDER 


3.2.2  Video  Processing  Computer 

The  video  processing  computer  is  used  to  perform  all  of  the  video  switching  and 
processing  functions  required  by  the  system.  It  is  configured  as  a  separate  machine 
because  of  the  large  amount  of  processing  that  will  be  required  to  perform  the  computer 
video  analysis  used  for  the  automatic  navigation  and  alignment  capabilities  which  will  be 
added  to  the  robot  in  later  phases  of  development.  Thus,  the  processor  resources  used 
for  image  analysis  are  independent  of  the  robot  control,  eddy-current  data  acquisition, 
and  the  user  interface.  A  photograph  of  the  video  processing  computer  is  presented  in 
Figure  10. 

3.2.3  Sateiiite  Equipment  Enciosure 

The  final  ground-based  unit  is  the  satellite  equipment  enclosure.  This  is  a  small,  mobile, 
19-inch  equipment  rack  used  to  house  much  of  ancillary  support  equipment  for  the 
system  such  as  power  supplies,  motor  controllers,  and  the  camera  control  units. 

All  electrical  lines  between  the  robot  and  the  ground  terminate  at  this  enclosure. 
Communication  and  eddy-current  signal  lines  are  then  routed  to  the  operator 
workstation,  while  video  lines  go  the  video  processor.  A  photograph  of  the  satellite 
equipment  enclosure  is  presented  in  Figure  1 1 . 
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FIGURE  10.  VIDEO  PROCESSING  COMPUTER 


FIGURE  11.  SATELLITE  EQUIPMENT  ENCLOSURE 


3.3  Hardware  Functional  Descriptions 
3.3.1  Communications  Subsystem 

The  ground-based  PC  communicates  with  both  the  motor  controllers  and  the  onboard 
computer  is  over  a  set  of  RS-232  serial  connections  as  shown  schematically  in  Figure 
12.  While  this  serial  communication  configuration  is  sufficient  for  use  in  the  lab,  it  is 
expected  that  a  combination  of  RS-422  and  optical  isolation  will  be  used  in  the  hangar  to 
support  longer  line  lengths  and  to  provide  greater  noise  immunity.  A  four  line  serial 
interface  is  used  to  provide  the  additional  communication  lines  from  the  PC.  Software 
drivers  enable  these  additional  lines  to  be  accessed  in  the  same  manner  as  standard 
PC  COM  ports.  The  16450  UART  ICs  on  the  communication  board  were  replaced  with 
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16550  UARTs  (with  on  chip  character  buffers)  to  provide  more  reliable  high-speed  serial 
communication  links  in  the  multitasking  environment. 


FIGURE  12.  COMMUNICATIONS  SUBSYSTEM 

The  stepper  motor  controllers  drive  the  lead  screw  motors  which  are  used  to  translate 
the  bridges  from  side  to  side.  The  linear  motor  controllers  drive  the  forcers  which  are 
used  to  move  the  bridges  along  the  spine.  Each  of  these  motor  controller  systems  is 
configured  in  a  daisy  chain,  allowing  multiple  controllers  to  share  a  single 
communication  line.  Commands  can  be  addressed  to  specific  controllers,  allowing 
independent  motion  control  and  status  reporting  for  each  bridge. 

The  operator  workstation  and  video  processor  PCs  are  also  connected  to  an  Ethernet 
local  area  network.  Communications  between  these  two  machines  use  standard  TCP/IP 
protocols  and  enable  high-speed  command  and  image  data  exchange.  This  connection 
also  provides  access  to  resources  on  other  computers  such  as  disk  storage,  printers, 
and  tape  drives.  Software  development  can  be  carried  out  on  multiple  computers  and 
then  easily  transferred  to  the  operator  workstation  as  completed.  This  data  path  can 
also  be  used  to  provide  access  to  the  X-Windows  user  interfaces  for  control  and 
monitoring  of  the  robot  through  remote  computers. 

3.3.2  Eddy-Current  Inspection  Subsystem 

Although  the  robot  is  capable  of  carrying  any  of  a  variety  of  small  NDI  sensors,  it  was 
decided  to  use  a  reflectance  sliding  eddy-current  probe  which  is  used  in  many  real-world 
inspections.  Deployment  of  this  type  of  device  has  historically  been  carried  out 
manually,  with  portable,  self-contained  instruments.  These  instruments  generally  do  not 
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have  the  electronic  interfaces  required  to  incorporate  them  into  larger  systems.  To 
embed  an  instrument  into  an  automated  data  collection  environment  requires  the  ability 
to  control  the  instrument  and  gather  data  through  an  electronic  interface  such  as  IEEE- 
488,  RS-232,  or  a  computer  bus,  e.g.  ISA 

An  ISA  bus  based  instrument  manufactured  by  SE  Systems,  Inc.,  the  smartEDDY™ 
3100,  was  selected  for  the  prototype  robotic  system.  This  instrument  is  capable  of  both 
reflectance  and  impedance  measurements  using  dual  inspection  frequencies  over  the 
range  from  1kHz  to  lOMhz.  It  is  installed  in  the  operator  workstation  computer  with  50- 
ohm  coaxial  cable  connections  to  the  Nortec  SPO-1 958/91 391 8  sliding  probe  installed 
on  the  robot  sensor  bridge. 

3.3.3  Video  Subsystem 

Video  imagery  provides  useful  information  in  both  direct  support  of  the  eddy-current 
inspection  as  well  as  improving  the  teleoperation  capability  of  the  robot  as  a  whole.  A 
total  of  four  video  cameras  are  currently  installed  on  the  robot  to  support  navigation  and 
alignment,  close-up  visual  rivet  inspection,  monitoring  of  the  robot,  and  large  area  visual 
inspection.  A  block  diagram  of  the  video  subsystem  is  provided  in  Figure  13. 

Two  Chinon  CX-060  miniature  black  and  white  video  cameras  are  used  for  navigation 
and  alignment  of  the  probe  with  respect  to  the  rivets  on  the  fuselage.  One  camera  is 
placed  at  each  end  of  the  robot’s  spine  assembly  looking  down  at  the  aircraft’s  surface. 
To  provide  additional  operational  feedback  to  the  operator,  two  Elmo  MN401 E  1/2-inch 
CCD,  (768H  X  494V  pixels),  color  cameras  were  also  placed  on  the  robot.  The  first  is 
the  “proprioception  camera”  which  is  mounted  at  the  front  end  of  the  robot  on  top  of  the 
computer  card  cage.  This  camera  has  a  wide  field  of  view  and  is  aimed  back  over  the 
robot  so  that  the  operator  may  view  the  robot  and  its  immediate  surroundings.  The 
second  color  camera  is  mounted  on  the  sensor  platform  and  is  aimed  at  the  eddy- 
current  probe.  This  camera  provides  a  detailed  visual  view  of  the  rivets  that  are  being 
inspected.  The  color  cameras  each  require  an  auxiliary  control  unit,  located  in  the 
satellite  equipment  enclosure,  and  a  multiconductor  control  cable. 

The  selection  of  the  currently  active  video  input  from  the  available  video  sources  is  done 
by  the  computer  using  the  video  multiplexer  which  has  twelve  video  inputs  and  seven 
video  outputs.  The  video  signal  is  digitized  by  the  EPIX  4MEG  VIDEO™  Model  12  video 
digitizer/processor.  This  card  has  4Mb  of  video  memory,  a  14.3  MHz  pixel  clock,  and  a 
dedicated  TMS320C25  12MIP  digital  signal  processing  chip.  It  is  capable  of  driving  an 
RGB  video  monitor  with  either  live  or  processed  video  imagery.  Both  of  these  cards  are 
installed  in  the  video  processing  PC. 

A  video  monitor  is  normally  used  to  display  the  video  data  to  the  operator.  However,  the 
capability  exists  to  display  the  video  data  in  a  subwindow  within  the  computer  monitor.  A 
video  recorder  is  provided  to  archive  the  video  data  from  the  cameras  as  well  as  to 
feedback  previously  recorded  data  into  the  system  for  the  testing  and  validation  of 
image  analysis  algorithms. 
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FIGURE  13.  VIDEO  SUBSYSTEM 


3.4  Software  Functional  Descriptions 

This  section  covers  the  processing  tasks  and  graphical  user  interfaces  shown  in  Figure 
3. 


3.4.1  Interprocess  Communications 

Coordination  of  the  various  software  processes  requires  them  to  exchange  information 
while  executing.  The  interprocess  communications  entity  shown  in  Figure  3  is  in  control 
of  all  communications  between  any  of  the  processing  tasks  or  between  the  graphical 
user  interfaces  and  the  processing  tasks.  It  currently  routes  text  or  binary  messages 
between  the  software  entities  executing  on  the  operator  workstation.  A  future 
development  goal  is  to  extend  the  capability  of  the  interprocess  communication  software 
to  include  other  hosts  on  the  LAN  through  the  use  of  the  TCP/IP  socket  level 
programming  model. 

3.4.2  Processing  Tasks 
3.4.2.1  Robot  Control 

The  robot  control  process  controls  all  higher  level  robot  functionality.  It  currently 
supports  the  following  operations: 

•  Walk  (forward,  back,  left,  right) 
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•  Scan  (forward,  back) 

•  Home  (calibrates  motor  position  by  sending  bridges  to  a  known  location) 

The  robot  control  process  provides  the  command  interface  to  the  motor  controllers  and 
the  onboard  PC  and  maintains  a  copy  of  the  current  robot  status. 

3.4.2.2  Eddy  Current 

While  a  self-contained  eddy-current  inspection  software  package  was  included  with  the 
SE  Systems,  Inc.  instrument,  it  was  structured  as  a  dedicated  application.  It  could  not 
exchange  information  with  other  parts  of  the  control  program,  and  it  required  the  entire 
capacity  of  the  machine.  To  integrate  the  eddy-current  instrument  with  the  robotic 
system,  a  custom  software  interface  was  written.  The  eddy-current  process  is  used  to 
set  the  parameters  of  the  instrument  such  as  inspection  mode,  signal  level,  signal 
frequency,  and  sampling  rate. 

The  eddy-current  process  also  performs  the  actual  acquisition  and  buffering  of  the 
inspection  data.  The  current  system  can  support  data  sampling  rates  of  approximately 
1  kHz.  Display  or  further  processing  of  the  data  is  left  to  the  eddy-current  user  interface. 

3.4.2.3  Video  Processing 

The  video  processing  application  runs  as  a  TCP/IP  network  server  on  the  video 
processing  PC.  It  sets  up  the  video  digitizer  and  multiplexer  hardware  and  then  listens 
on  the  local  area  network  for  commands  from  the  operator  workstation  PC.  Video  data 
may  be  acquired  and  displayed  from  multiple  sources,  saved  to  or  restored  from  disk, 
and  overlaid  with  an  alignment  cursor. 

Overlaying  the  video  from  the  front  and  rear  alignment  cameras  with  a  calibrated  cursor 
enables  the  operator  to  manually  align  the  robot  with  the  rivet  rows.  This  function 
enables  manual  operation  of  the  robot  in  the  short  term  while  the  automatic  alignment 
algorithms,  described  in  Section  3.4.4.2,  are  developed  and  integrated  into  the  system. 

3.4.3  User  Interfaces 

Graphical  user  interfaces  were  developed  to  simplify  the  use  of  the  system  and  the 
display  of  inspection  data.  Controls  are  provided  using  a  combination  of  pull-down 
menus,  push  buttons,  text  fields,  and  command  lines.  Feedback  from  these  actions  are 
rapidly  provided  through  visual  indication  of  the  robot’s  status,  graphs  representing 
eddy-current  data,  and  wireframe  models  of  the  robot  and  the  aircraft  being  inspected. 

The  X-Windows  interface  was  chosen  for  implementation  of  the  robot  system  due  to  its 
portability  to  multiple  types  of  computers  and  operating  systems.  The  interface  was 
implemented  using  a  client/senrer  model  and  is  usable  over  a  network.  This  allows 
multiple  computers  to  transparently  share  inspection  subtasks  and  results.  The  use  of 
X-Windows  also  allows  multiple  interfaces  to  be  simultaneously  displayed  in  separate 
windows  on  a  single  computer  monitor.  Samples  of  the  user  interfaces  are  presented  in 
Figures  1 4  through  22. 
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3.4.3.1  Robot  Control 


Control  of  the  robot  and  feedback  of  its  status  is  provided  through  the  control/debug, 
router,  and  message  interfaces. 

The  control/debug  interface,  Figure  14,  provides  interaction  with  the  robot  graphically, 
through  a  schematic  representation  of  the  physical  robot  hardware.  The  spine  and  both 
bridges  are  presented.  Individual  control  points  are  accessible  via  on/off  or  arrow  push 
buttons.  Selecting  these  buttons  with  the  mouse  sends  commands  to  the  onboard 
computer  to  turn  pneumatic  valves  on  or  off,  or  to  move  the  motors.  Status  information 
obtained  from  the  robot  is  indicated  through  color  changes  in  the  display: 

•  control  points,  (On=White,  Off=Gray) 

•  limit  switches,  (On=Yellow,  Off=Gray) 

•  suction  cups,  (Vacuum=Green,  Ambient  pressure=Red) 


FIGURE  14.  ROBOT  CONTROL  INTERFACE 


The  ANDI  command  interface,  shown  in  Figure  15,  sends  textual  commands  to  any  of 
the  cooperating  interfaces  or  processes  through  the  router  process.  This  provides 
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support  for  both  system  debugging  as  well  as  more  complicated  motion  sequences  run 
from  the  control  process.  These  moves  include  walking,  scanning,  and  homing  the 
robot’s  motors. 


For  system  debugging  purposes,  low-level  messages  from  the  onboard  computer  are 
displayed  in  the  communication  screen,  Figure  16.  The  current  status  of  each  of  the 
robot’s  inputs  and  output  are  divided  into  groups  of  eight  bits,  forming  data  bytes  which 
are  displayed  using  hexadecimal  notation.  There  are  currently  five  input  and  five  output 
bytes  used  to  reflect  the  robot’s  status.  Output  transitions,  text  messages  generated  by 
tasks,  and  text  responses  to  debugging  commands  are  also  displayed  here. 


FIGURE  15.  COMMAND  INTERFACE 


FIGURE  16.  COMMUNICATION  DISPLAY 


3.4.3.2  Eddy-Current  Instrument  Control 

The  eddy-current  control  interface  is  used  to  control  the  inspection  parameters  of  the 
eddy-current  instrument,  as  well  as  to  display  the  acquired  data. 
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The  inspection  parameters  that  can  be  set  in  the  eddy-current  control  screen  in  Figure 
17  include: 


A  and  B  channel  oscillator  frequencies  and  levels 

reflectance  or  impedance  inspection  mode  via  the  signal  selection  relays 

data  sampling  rate 


FIGURE  17.  EDDY-CURRENT  INSTRUMENT  INTERFACE 


Acquired  data  may  be  displayed  to  the  operator  or  also  archived  to  a  file  for  later 
analysis.  The  data  can  be  presented  in  both  textual,  shown  in  Figure  18,  and  graphical 
forms.  The  graphical  displays  can  plot  the  data  as  In-phase  vs.  Quadrature  signals. 
Figure  19,  or  as  separate  In-phase  vs.  Time  and  Quadrature  vs.  Time  plots.  Figure  20. 
Each  frequency  channel  is  displayed  in  a  different  color  and  the  operator  has  the  ability 
to  zoom  in  on  selected  sections  of  the  plots  to  view  the  data  in  greater  detail. 

There  is  currently  no  automatic  flaw  detection  capability  in  the  eddy-current  software. 
Simple,  threshold-based  flaw  detection  algorithms  may  be  incorporated  to  more  fully 
demonstrate  the  autonomous  operation  of  the  inspection  system. 


3.4.3.3  Video  Control 

The  video  control  interface  is  used  by  the  operator  to  manually  adjust  the  robot 
alignment  prior  to  eddy-current  inspection.  It  consists  of  both  text  and  graphically  based 
commands  and  is  shown  in  Figure  21.  Text-based  scripts  are  used  to  send  commands 
to  the  video  processing  PC  to  select  the  active  video  input  and  to  overlay  the  video 
cursor  for  the  front  and  rear  alignment  cameras. 

The  graphical  user  interface  is  used  to  set  the  robot  in  the  alignment  mode  with  legs 
extended  and  pivot  locks  off.  The  operator  can  then  rotate  and  jog  the  robot  to  bring  the 
eddy-current  sensor  into  the  correct  position  for  scanning  by  depressing  the  mouse 
button  and  dragging  the  mouse  while  the  cursor  is  in  the  black  rectangular  region  of  the 
display.  Once  properly  aligned,  the  robot  is  returned  to  scanning  mode  with  legs 
retracted  and  pivot  locks  on. 
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FIGURE  18.  EDDY-CURRENT  TEXTUAL  DATA  DISPLAY 


FIGURE  19.  EDDY-CURRENT  IMPEDANCE  PLANE  DISPLAY 
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FIGURE 20.  EDDY-CURRENT  IN-PHASE/QUADRATURE  VS.  TIME  DISPLAY 


3.4.3.4  Aircraft/Robot  3-D  Map 

A  prototype  graphical  display  of  the  robot’s  inspection  path  was  created  using  a  three- 
dimensional  wire-frame  representation  of  the  surface  of  the  airframe  under  inspection. 
While  CAD  data  obtained  directly  from  the  aircraft  manufacturer  or  the  airlines  would  be 
preferred  for  use  as  the  basis  for  aircraft  maps,  it  is  typically  not  available  for  the  older 
generation  aircraft  now  being  inspected.  An  alternative  is  the  use  of  models  of  various 
aircraft  which  are  available  from  computer  animation  companies.  While  these  models 
are  optimized  for  their  visual  representation  and  do  not  generally  include  structural 
information,  they  do  provide  a  sufficient  framework  to  provide  navigation  information  at 
the  resolution  required  by  the  robotic  system.  The  frame  and  stringer  or  longeron 
intersections  serve  as  navigation  points  whose  approximate  location  can  be  obtained 
from  service  documentation  and  measurements  from  the  aircraft.  These  intersection 
points  define  the  lines  where  rivets  are  found  and  can  be  manually  fit  to  the  aircraft 
model  using  computer  aided  drafting  tools.  Individual  rivets  are  not  modeled.  In 
addition  to  the  ability  to  model  aircraft,  information  concerning  test  fixtures,  such  as  the 
Foster  Miller  laboratory  skin  panel,  are  easily  input.  A  sample  display  of  the  lines 
connecting  the  frame  and  stringer  intersections  on  the  Foster  Miller  test  panel  used  in 
the  laboratory  is  provided  in  Figure  22. 

The  operator  currently  has  the  ability  to  import  various  map  data  files  into  the  viewport 
and  modify  the  point  of  view.  Rivet  lines  are  indicated  in  a  contrasting  color.  Eventually, 
the  operator  will  be  able  to  define,  via  an  input  device  such  as  a  mouse,  the  robot  path 
by  selecting  appropriate  lines  of  rivets  for  inspection.  Areas  that  are  selected  for 
inspection  or  have  been  inspected  or  where  anomalies  have  been  found  may  all  be 
indicated  through  the  color  used  to  display  the  segment.  It  is  also  possible  to  overlay  an 
image  of  the  robot  on  this  display  to  monitor  the  position  of  the  robot  on  the  aircraft 
being  inspected. 
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FIGURE  21.  VIDEO  CONTROL  INTERFACE 


FIGURE  22.  AIRCRAFT  MAP  DISPLA  Y  (FOSTER  MILLER  PANEL) 


3.4.4  Miscellaneous 

There  are  two  additional  software  efforts,  detailed  in  the  following  sections,  that  are 
supporting  the  robotic  system’s  long  term  development.  Techniques  developed  through 
this  research  will  be  incorporated  into  the  robotic  system  as  appropriate. 

3.4.4.1  Robot  Animation 

The  U.S.  Bureau  of  Mines  has  developed  a  three-dimensional  animation  and  rendering 
of  the  robot  and  aircraft  surface.  This  tool,  which  runs  on  a  Silicon  Graphics 
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workstation,  enables  the  visualization  of  the  robot’s  motion  and  its  interaction  with  the 
surface  being  inspected.  The  animation  of  the  robot  can  be  controlled  using  either 
predefined  scripts  of  motion  commands  or  via  the  X-Windows  robot  controi  interface 
described  in  Section  3.4.3.1. 

3.4.4.2  Video  Rivet  Location  Algorithms 

To  allow  the  robot  to  align  the  eddy  current  probe  to  the  rivets  undergoing  inspection  as 
well  as  to  navigate  autonomously  over  the  aircraft's  surface  requires  algorithms  to 
detect  the  presence  and  measure  the  location  of  rivets  in  the  video  imagery  from  the 
navigation  cameras,  and  also  to  abstract  from  the  detected  isolated  rivets  the  imaginary 
"lines  of  rivets"  that  form  the  natural  reference  grid  for  navigation  and  the  natural 
scanning  paths  for  the  eddy  current  inspection  probe.  Algorithms  to  do  this  rivet 
detection  and  rivet  line  abstraction  were  developed  at  the  CMU  Robotics  Institute.  A 
demonstration  of  rivet  image-based  alignment  with,  and  navigation  along,  a  grid  of  rivet 
holes  in  a  sample  of  DC-9  belly  skin  was  performed  and  videotaped  to  verify  the  ability 
of  the  algorithms  to  function  correctly  in  a  closed  loop  control  system.  The  rivet  finding 
and  rivet  line  abstraction  algorithms  will  be  transferred  into  ANDI's  video  processing 
system  during  the  next  phase  of  the  project.  The  key  features  of  the  approach  and 
demonstration  are  summarized  in  the  following  paragraphs  and  detailed  in  a  conference 
paper[3]. 

If  the  rivets  that  hold  an  aircraft's  skin  to  its  frame  all  looked  the  same,  then  it  would  be 
easy  to  construct  a  dedicated  algorithm  that  would  reliably  find  and  mark  all  the  rivets  in 
an  image.  However  there  are  many  types  of  fasteners  in  use  on  new  aircraft,  many 
more  types  in  use  on  in-service  aircraft,  and  even  identical  fasteners  on  the  same 
aircraft  take  on  different  appearances  depending  on  the  details  of  their  installation, 
location,  weathering,  buffing,  coating,  etc.  To  accommodate  this  variation  range,  we 
adopted  an  open  ended  approach  using  a  neural  network  architecture  than  learns  from 
visual  examples  presented  to  it  and  classified  by  a  human  "trainer"  as  containing  or  not 
containing  rivets.  The  visual  examples  are  small  square  "windows"  on  the  full  image;  a 
window's  side  is  about  twice  the  diameter  of  the  largest  rivet  head  that  will  be 
encountered.  Thus  the  window  may  contain  all  or  part  or  none  of  a  rivet  head,  but  it  will 
rarely  contain  more  than  one. 

In  operation,  the  trained  neural  network  generates  a  numerical  output,  that  we  call 
"rivetness",  that  represents  its  estimate  of  the  probability  that  the  window  it  is  being 
shown  contains  a  rivet.  The  window  is  scanned  over  an  image  resulting  in  a  "rivetness" 
map  that  is  bright  in  areas  where  rivets  are  probable,  dark  in  areas  where  they  are 
improbable,  and  gray  in  ambiguous  areas.  A  sample  of  the  “rivetness”  image  is 
provided  in  Figure  23.  Via  several  subsequent  conventional  image  processing  steps  the 
"rivetness"  image  is  converted  to  a  black-and-white  (binary)  image  in  which  rivets  are 
white,  non-rivets  are  black.  The  binary  image  obtained  from  Figure  23  is  shown  in 
Figure  24.  The  binarization  process  completes  the  rivet  finding  step. 

Next,  it  is  necessary  for  the  computer  to  abstract  isolated  rivets  into  rivet  lines  and  a 
rivet  line  grid;  this  kind  of  abstraction,  which  is  the  natural  thing  for  the  human  brain  to 
do  when  it  is  presented  with  isolated  dots  that  happen  to  fall  on  lines  and  rectilinear 
grids,  is  a  potentially  difficult  task  for  a  computer.  It  is  a  simple  and  very  common 
procedure  to  fit  a  line  to  a  set  of  data  points  in  some  coordinate  system,  e.g.,  by  the 
least  squares  method,  but  exactly  where  this  line  falls  can  be  very  strongly  influenced  by 
a  small  number  of  points  that  are  far  from  the  main  distribution;  that  is,  these  methods 
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are  inappropriately  sensitive  to  outliers.  Any  image  containing  a  line  of  rivets  will 
probably  also  contain  a  few  extraneous  rivets  that  would  be  effortlessly  rejected  as 
irrelevant  by  any  human  but  that  would  not  be  recognized  as  irrelevant  by  conventional 
line  fitting  algorithms.  Fortunately,  modern  statisticians  have  recognized  this  problem, 
and  in  response  they  have  developed  robust  line  fitting  methods  that  do  an  excellent  job 
of  focusing  on  the  main  stream  and  ignoring  the  outliers.  These  algorithms  have  been 
extremely  successful  in  identifying  rivet  lines.  The  line  in  Figure  24  is  the  output  of  the 
robust  line  fitting  algorithm. 


FIGURE  23.  RIVETNESS  IMAGE 

FIGURE  24.  BINARY  IMAGE  WITH  LINE 


An  existing  Robotics  Institute  laboratory  robot,  that  was  built  as  a  development  platform  for 
enhanced  visual  inspection  sensors,  was  used  to  test  the  rivet  detection  and  line  fitting 
algorithms.  The  “test  inspection  platform"  (TIP)  robot  is  capable  of  pure  rotation  (with  negligible 
forward/backward  or  left/right  motion),  so  it  could  follow  the  requested  path  precisely  despite 
the  path's  right  angle  corners.  It  was  guided  around  a  rectangular  path  under  the  closed  loop 
control  of  the  algorithms.  To  provide  a  rapid  demonstration-of-principle,  we  assembled  a 
multicomputer  pipeline  in  which  the  tasks  of  image  collection,  image  transmission,  rivet 
detection  and  rivet  line  abstraction,  robot  control  strategy  generation,  robot  control  command 
generation,  and  robot  control  command  execution  were  done  on  several  different  workstations, 
in  several  different  locations,  communicating  over  the  campus  Ethernet  LAN.  Although  the 
decision  making  process  was  slow,  primarily  because  of  the  need  to  move  large  image  files 
over  the  Ethernet,  the  navigation  sequence  was  executed  without  error,  the  path  following 
accuracy  was  excellent,  and  the  TIP  robot  always  discovered  and  corrected  random  orientation 
errors  that  the  top-level  program  introduced  to  challenge  the  lower-level  programs. 
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4.  SYSTEM  TESTING  AND  FIELD  DEMONSTRATION 


A  panel  fixture  to  support  a  simulated  aircraft  panel  was  designed  and  fabricated  for  testing  the 
robot  in  the  laboratory.  This  fixture  allows  the  panel  to  be  rotated  360°  and  locked  into  any 
desired  position  for  testing.  The  panel  fixture,  shown  in  Figure  25,  allows  the  robot  to  be  tested 
in  all  orientations,  simulating  the  top,  bottom,  and  sides  of  an  aircraft.  A  trolley  runs  above  the 
panel  for  tethering  the  robot.  The  robot  can  be  seen  sitting  on  a  cabinet  in  the  foreground  of 
Figure  25. 


FIGURE  25.  LABORATORY  TEST  PANEL  FIXTURE 

ANDI  was  demonstrated  at  the  Air  Transport  Association's  Nondestructive  Testing  Forum  on 
November  2, 1994,  at  the  FAA  Aging  Aircraft  NDI  Validation  Center  (AANC)  in  Albuquerque, 
NM.  Prior  to  this  demonstration,  extensive  laboratory  and  hangar  tests  were  conducted.  A 
picture  of  the  robot  walking  on  the  DC-9  nose  section  is  shown  in  Figure  26. 

This  remainder  of  this  section  outlines  the  issues  uncovered  during  the  period  of  laboratory 
testing  and  preparation  for  the  field  demonstration. 

4.1  Linear  Motors 

Several  problems  associated  with  the  linear  motors  were  found  during  system  testing.  The 
linear  motors  tended  to  stall  following  1/2  to  3/4  hours  of  use.  The  motors  generate  heat  when 
they  are  used,  and  there  is  no  efficient  way  to  dissipate  the  heat.  Because  the  platen  (spine) 
was  machined  into  a  U-channel  to  save  weight,  there  is  very  little  material  for  heat  conduction. 
This  lack  of  material  most  likely  causes  a  thermal  expansion  of  the  platen,  and  over  time,  the 
platen  expands  enough  to  cause  the  motors  to  stall. 

Severe  wear  was  also  noted  on  the  linear  motor  platen;  this  may  have  two  causes.  First,  the 
thermal  expansion  of  the  platen  may  cause  wear  on  the  platen  before  the  motors  stall.  Also, 
because  the  platen  was  machined  into  a  U-channel,  it  may  not  be  perfectly  flat,  causing  the 
motors  to  rub  against  the  surface  of  the  platen,  creating  wear.  The  Stabilizer  Bridge  did  not 
move  freely  even  when  the  system  was  cold,  indicating  that  the  spine  may  not  be  uniformly  flat. 
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Finally,  there  was  a  loss  of  motor  position  calibration  after  either  of  the  bridges  ran  into  a  hard 
stop  or  after  the  bridges  ran  into  each  other.  If  this  happened,  the  motors  had  to  be  sent  back 
to  their  home  positions  for  proper  initialization. 

Enough  problems  with  the  linear  motors  were  uncovered  that  it  might  be  best  to  explore  other 
options  for  creating  linear  motion  along  the  spine. 


FIGURE  26.  ROBOT  ON  DC-9  NOSE  SECTION 


4.2  Lead  Screw  Assemblies 

The  lead  screw  assemblies  that  provide  linear  translation  in  the  direction  perpendicular  to  the 
spine  were  slow,  on  the  order  of  1 0  inches  per  minute.  When  the  motors  were  run  at  higher 
speeds,  they  did  not  provide  enough  torque  for  motion.  The  installation  of  higher  torque  motors 
for  the  same  assemblies  provided  only  marginally  greater  torque  at  higher  speeds,  and  speed 
was  not  measurably  increased.  This  issue  needs  further  investigation. 

4.3  Umbiiical/Tether  Management 

The  problems  inherent  with  managing  both  a  thick  umbilical  cable  and  a  tether  are  operational 
issues  that  must  be  solved  before  surface-crawling  robots  can  effectively  be  used  in  a 
commercial  aircraft  inspection  facility.  Although  these  issues  are  beyond  the  scope  of  this 
project,  it  was  apparent  from  the  field  demonstration  that  tether  and  umbilical  management  are 
critical. 

The  umbilical  is  the  line  that  extends  from  the  control  area  to  the  robot.  Presently,  the  umbilical 
comprises:  three  1 /4-inch  pneumatic  lines,  two  eddy-current  cables,  four  video  cables,  four 
motor  control  cables,  a  power  cable,  and  a  computer  communications  cable.  The  diameter  of 
this  bundle  is  about  1-1 /4-inches. 

During  the  demonstration,  the  robot  walked  over  the  crown  of  the  nose  section  of  a  McDonnell 
Douglas  DC-9  (the  section  that  stretches  from  the  nose  of  the  aircraft  to  just  past  the  second 
passenger  window).  For  this,  the  length  of  the  umbilical  cable  was  50  feet;  it  was  very  difficult 
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to  manage  a  cable  of  that  thickness  and  length.  To  inspect  the  entire  fuselage  of  a  DC-9  or 
other  aircraft,  the  robot  would  require  a  much  longer  cable.  If  the  robot  were  to  inspect  a  wide- 
body  aircraft,  several  hundred  feet  of  umbilical  cable  would  be  required.  If  robotics  is  to  be 
practical  in  a  commercial  hangar  environment,  the  umbilical  must  be  either  made  more 
manageable  or  perhaps  eliminated  altogether. 

The  tether  for  the  demonstration  ran  along  a  safety  line  suspended  above  the  DC-9:  however, 
the  tether  did  not  move  freely  enough  to  gracefully  move  in  concert  with  the  robot.  There  were 
times  when  the  tether  became  entangled  with  the  bridges,  thus  interfering  with  the  robot's 
motion.  In  the  future,  coordinated  motion  between  the  tether  and  the  robot  may  be  necessary. 
An  active  tether  whose  motion  would  be  in  sync  with  the  robot  is  an  area  of  potential  research 
for  the  future. 

4.4  Navigation 

During  the  field  trials,  it  was  noted  that  navigation  around  seams  was  complicated  by  the 
spacing  and  size  of  the  suction  cups  and  the  3-1/2-inch  sideways  step  size  of  the  bridges. 

There  are  several  potential  solutions  to  this  problem.  First,  clusters  of  suction  cups  could  be 
added  to  each  leg;  this  would  have  the  disadvantage  of  increasing  the  amount  of  air  used  by 
the  system.  A  second  potential  solution  is  to  adjust  the  dimensions  of  the  bridges  to  make 
navigating  around  seams  easier. 

4.5  Walking  Motion 

The  walking  motion  of  the  robot  created  some  operational  problems.  On  inverted,  angled 
surfaces,  the  extend  and  retract  motion,  in  which  the  spine  is  raised  from  and  lowered  to  the 
surface,  was  unreliable.  This  was  primarily  due  to  a  combination  of  the  weight  of  the  robot  and 
the  sizes  of  the  pneumatic  lifting  cylinders.  Also,  the  extend  and  retract  motion  severely  jars  the 
mechanical  system,  loosening  and  breaking  electrical  contacts  and  shifting  camera  views.  A 
third  problem  with  the  walking  motion  is  that  the  variation  in  the  robot's  position  between  the 
extended  and  retracted  states  makes  alignment  difficult.  Alignment  is  performed  in  the 
extended  state  while  scanning  occurs  in  the  retracted  state.  One  apparent  solution  for  these 
problems  is  to  replace  the  legs  on  the  spine  assembly  with  legs  that  can  extend  and  retract. 
During  walking,  the  spine  would  stay  at  a  uniform  height  while  the  individual  legs  would  be 
raised  and  lowered. 

4.6  Surface  Damage 

During  field  testing,  it  was  noted  that  the  suction  cups  left  scuff  marks  on  the  aluminum  skin; 
the  situation  was  much  worse  on  the  Foster  Miller  panel  than  on  the  DC-9  fuselage  surface. 

This  problem  may  be  connected  with  the  dirt  on  the  panel  and  fuselage  surfaces.  Even  though 
the  surfaces  had  been  cleaned,  a  film  of  fine  dirt  particles  remained  on  both  surfaces.  As  the 
robot  walked  along  either  surface,  the  dirt  particles  collected  on  the  suction  cups.  As  the  cups 
attached  to  and  released  from  a  surface,  they  tended  to  slide  a  little,  especially  when  the  spine 
assembly  was  lowered  to  the  surface.  The  dirt  trapped  on  the  bottom  surface  of  the  suction 
cups  scored  the  surface  slightly  as  the  cup  slid.  A  potential  solution,  suggested  by  a 
representative  of  an  aluminum  producer,  is  the  replacement  of  the  soft  silicone  suction  cup  with 
a  harder  rubber  that  would  be  less  likely  to  trap  dirt. 

In  a  related  problem,  the  eddy-current  probe  did  not  initially  ride  flat  on  the  skin  surface, 
compromising  scan  data.  To  compensate  for  this,  the  pressure  used  to  deploy  the  sensor  was 
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increased.  As  with  the  suction  cups,  dirt  accumulated  on  the  probe’s  surface,  and  when  it  was 
deployed,  the  probe  also  scored  the  skin  surface. 

The  surface  abrasion  caused  by  the  robot  did  not  result  in  deep  scratches  to  the  surfaces  and 
could  easily  be  buffed  away.  However,  this  may  not  be  acceptable  to  aircraft  maintenance 
personnel  and  must  be  further  investigated.  The  tendency  to  score  an  aluminum  surface  is  a 
characteristic  of  all  surface-walking  robots.  Dirty  surfaces  will  be  regularly  encountered  during 
routine  inspections  in  a  real  environment,  so  this  problem  must  be  addressed  before  surface¬ 
walking  robots  can  be  routinely  deployed  in  the  field. 
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5.  CONCLUSIONS 


Through  laboratory  testing  at  CMU  and  field  testing  at  the  AANC  in  Albuquerque,  New  Mexico, 
the  technical  feasibility  of  the  robot  system  has  been  demonstrated.  Specifically,  the  robot  was 
able  to  achieve  the  goals  of: 

•  adhering  to  an  aircraft  fuselage 

•  walking  over  its  surface 

•  acquiring  eddy-current  inspection  data. 

In  an  auxiliary  experiment  using  another  robot  moving  on  an  aircraft  skin  surface,  video 
camera-based  automatic  alignment  and  navigation  were  demonstrated. 

A  significant  development  effort  remains  before  this  robotic  inspection  technology  is  suitable  for 
operational  deployment.  Outstanding  development  issues  include: 

•  improving  the  mechanical  reliability  and  speed  of  the  system 

•  reduction  or  elimination  of  the  umbilical  cable 

•  automation  of  many  of  the  manually  controlled  operations 

While  an  extensive  redesign  of  the  mechanical  system  will  be  required  to  obtain  a  commercial 
version  of  the  robot,  the  existing  robot  provides  an  adequate  platform  to  continue  development 
of  the  control  software  for  further  automation  of  the  inspection  process.  The  integration  of  the 
video-based  rivet  detection  algorithms  with  the  robot  control  software  and  the  enhancement  of 
the  robot  navigation  and  control  software  are  crucial  to  making  this  technology  succeed  and  will 
be  the  focus  of  the  next  phase  of  the  project.  Once  these  issues  are  resolved,  designing  and 
building  a  new  mechanical  system  to  address  the  speed  and  reliability  of  the  robot  must  be 
undertaken  to  commercialize  the  technology. 
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